Breaking Changes:

This PR renames RIKOLTI_ES_ENDPOINT to OPENSEARCH_ENDPOINT - please update env.local and dags/startup.sh accordingly. We will need to update MWAA's startup.sh file too.

Updates:

Adds a docker-compose file to spin up a local dev opensearch instance:

cd record_indexer
docker-compose up

Updates the README with relevant details regarding connecting to either this dev opensearch instance or an AWS hosted instance.

Adds an initialization script initialize/add_rikolti-stg_index.py - this script and index_templates/rikolti_template.py must be run to initialize this dev opensearch instance, before it can be used with any of the Rikolti record_indexer functions.

Non-breaking Removals:

• INDEX_RETENTION and RIKOLTI_ES_ALIAS_STAGE - you can remove these from your env.local and dags/startup.sh files. Since we're no longer creating an index per collection per harvest process, we no longer have need for an index retention variable, and we are using rikolti-stg as more of a special constant than a variable alias.
• create_stage_index_task used to create a new index for a collection, index a collection's documents into it, and assign the rikolti-stg alias to the index.
• move_index_to_prod module and index_to_prod dag used to assign the rikolti-prd alias to a collection's index.
• create_collection_index used to provide a bunch of functions for creating/deleting indices.